1. Field of the Invention
The present invention relates to an image processing apparatus, such as an image data compressing or expanding apparatus, and an image processing method which is employed in such an apparatus.
2. Related Background Art
Regarding the images created on a computer by, for example, DTP, there is a demand for a higher image quality in recent years, and development of color or multi-level images has advanced. The quantity of data in such an image quality is about 88 M bytes in the case of an image having, for example, A4 size, 4000 dpi, and 256 xe2x80x9cgrayxe2x80x9d levels (gradations) for each of three colors.
Handling image data as code data, such as in a page description language, can reduce the quantity of data. However, it takes time for code data to be transformed into image data. In addition, it may be impossible to reproduce original image data from the transformed code data.
Color multilevel images are generally compressed by the ADCT (adaptive discrete cosine transform) method recommended by JPEG (Joint Photographic Expent Group). This ADCT compression method will be described below in detail.
FIG. 26 is a block diagram illustrating a functional configuration of an ADCT compressing apparatus.
In the figure, reference numeral 3101 denotes a color space conversion unit for converting a color space (NTSC-RGB) by signals of respective colors (red (R), green (G) and blue (B)) of the NTSC method into a color space (YCrCb) represented by a luminance signal Y and two color difference signals Cr and Cb.
Reference numeral 3102 denotes a sub-sampling unit for reducing color difference data by utilizing the characteristics of human eyes in that they are sensitive to a luminance and insensitive to a color difference. In a practical operation, the sub-sampling unit 3102 reduces the quantity of color difference data to one half of the original quantity by obtaining the average value of adjacent two color difference data. Reference numeral 3103 denotes a DCT unit for dividing the image data input through the sub-sampling unit 3102 into blocks each having dimensions of 8xc3x978 in the horizontal and vertical directions and for conducting DCT on each of the obtained blocks to transform the input image into a frequency space.
Reference numeral 3104 denotes a quantization unit for dividing 64 DCT coefficients by quantized values having different step widths. Reference numeral 3105 denotes a Huffman encoding unit for dividing the 64 quantized DCT coefficients into a single DC coefficient and 63 AC coefficients and for coding the respective coefficients according to the Huffman table recommended by JPEG. After a header, such as quantization table data or Huffman table data, is affixed to the coded data, the coded data is stored in a memory or transmitted to another apparatus.
FIG. 27 is a block diagram illustrating a function configuration of an ADCT expanding apparatus.
In the figure, reference numeral 3205 denotes a Huffman decoding unit for decoding input coded data to generate quantized data. Reference numeral 3204 denotes an inverse quantization unit for transforming the quantized data generated by the Huffman decoding unit 3205 into DCT coefficient data. This is achieved by multiplying 64 coefficients by the quantized values using the quantization table data employed by the quantization unit 3104 for quantization.
Reference numeral 3203 denotes an inverse DCT unit for performing inverse DCT on the DCT coefficient data obtained by the inverse quantization unit 3204 to obtain actual image data. Reference numeral 3202 denotes an interpolation unit for interpolating the data Cr and Cb which have been removed by the sub sampling unit 3102 during data compression by the simple repeating method. Reference numeral 3201 denotes a color space editing unit for converting the data YCrCb into NTSC-RGB data or color space data suitable for that particular apparatus.
The flow of the process in a conventional data compressing or expanding apparatus will now be described.
FIG. 28 illustrates part of image data of a color multilevel image created on a computer. The image data shown in FIG. 28 is NTSC-RGB data which shows the portion (16 pixelsxc3x9716 pixels) of the image data representing a character portion of a color multilevel image.
In FIG. 28, reference numeral 3301 denotes red (R) data, reference numeral 3302 denotes green (G) data, and reference numeral 3303 denotes blue (B) data. The data of each pixel is obtained by a 8-bit register (0 to 255). The data shown in FIG. 28 represents the portion of an image in which a character of blue (R, G, B)=(34, 30, 225) is written on a background of a slightly dark white (R, G, B)=(225, 225, 225).
The color space conversion unit 3101 performs conversion from NTSC-RGB to YCrCb by the following equations:
Y=0.299xc3x97R+0.587xc3x97G+0.114xc3x97B
Cr=0.713 (Rxe2x88x92Y)
Cb=0.564 (Bxe2x88x92Y)
In addition, in order to allow for overshoot or undershoot during the operations in the YCrCb color space, the color space conversion unit 3101 performs rounding of data in conformity with the recommendation of CCIR as follows:
Y=219.0xc3x97Y+16.5
Cr=224.0xc3x97Cr+128.5
Cb=224.0xc3x97Cb+128.5
The sub-sampling unit 3102 performs sub-sampling on the color difference data Cr and Cb in the data of YCrCb obtained by the above-described equations using the average value method as the sub-sampling method. In this method, the average value of the data of the adjacent two pixels is obtained to obtain single data.
FIG. 29 illustrates the data obtained through the color space conversion unit 3101 and the sub-sampling unit 3102. In FIG. 29, reference numeral 3401 denotes Y data (luminance data), reference numeral 3402 denotes Cr data (color difference data) and reference numeral 3403 denotes Cb data (color difference data). As can be seen in FIG. 29, the quantity of each of Cr and Cb data has been reduced to one half by the sub-sampling unit 3102.
Y data 3401, Cr data 3402 and Cb data 3403, shown in FIG. 29, are input to the DCT unit 3103. The DCT unit 3103 divides the data into blocks each of which contains data having dimensions of 8xc3x978 in the horizontal and vertical directions. As a result, Y data 3401 is divided into four blocks 3401a through 3401d. Similarly, Cr data 3402 is divided into two blocks 3402a and 3402b. DCT is performed on each of these eight blocks.
FIG. 30 illustrates the data obtained by performing DCT conversion on the eight blocks shown in FIG. 29. In FIG. 30, reference numeral 3501 denotes data obtained by performing DCT on Y data 3401. The four blocks 3401a through 3401d respectively correspond to blocks 3501a through 3501d. Similarly, reference numeral 3502 denotes data obtained by performing DCT on Cr data 3402. Reference numeral 3503 denotes data obtained by performing DCT on Cb data 3403. 64 coefficients of each of the blocks obtained by DCT include a single DC component (in the left upper corner) and 63 AC components.
Next, the quantization unit 3104 quantizes the DCT data 3501 through 3503 shown in FIG. 30.
The quantization table recommended by JPEG is employed as the quantization table for quantization. FIG. 35 illustrates that quantization table. In FIG. 35, reference numeral 4001 denotes data for Y components, and reference numeral 4002 denotes data for Cr and Cb components. FIG. 31 illustrates data obtained by quantization. Reference numeral 3601 denotes data obtained by quantizing Y data. Reference numeral 3602 denotes data obtained by quantizing Cr data. Reference numeral 3603 denotes data obtained by quantizing Cb data.
The Huffman coding unit 3105 divides each of the quantized data 3601 through 3603 into a DC component and AC components. Regarding the DC component, the Huffman coding unit 3103 creates the optimum Huffman coding table by obtaining a histogram for a difference between the DC component of a previous block and the DC component of the present block, and performs coding according to that table.
The AC components are re-arranged zigzag, as shown in FIG. 36. The Huffman coding unit 3105 creates the optimum Huffman table by obtaining a histogram for a combination of a run length of xe2x80x9c0xe2x80x9d obtained before a value xe2x80x9cXxe2x80x9d, other than xe2x80x9c0xe2x80x9d, enters and the value xe2x80x9cXxe2x80x9d, and performs coding according to the created optimum Huffman table.
By this time, the NTSC-RGB data of 16xc3x9716 pixels have been transformed to data having 796 bits. Since a single original pixel has 8 bits, 16xc3x9716xc3x97three colors=786 bytes=6144 bits. Therefore, compression of 1/7.7 has been performed. In an actual operation, an image size, the quantization table and the coding table are affixed to the obtained coded data, and that coded data is stored in a memory or transmitted.
The flow of the process in a conventional data expanding apparatus will be described below.
The image data processed by the ADCT compression method enters and are decoded by the Huffman decoding unit 3205. The inverse quantization unit 3204 performs inverse quantization on the data decoded by the Huffman decoding unit 3205 by multiplying the obtained data by the coefficients in the quantization table shown in FIG. 35.
Thus, the data shown in FIG. 32 is obtained. The data shown in FIG. 32 is obtained by performing Huffman decoding and inverse quantization on the data obtained by the ADCT compression method. As is clear from FIG. 32, the data shown in FIG. 32 is completely different from the data shown in FIG. 30 which is not yet compressed.
The inverse DCT unit 3202 transforms the data shown in FIG. 32 into YCrCb data. FIG. 33 illustrates the obtained YCrCb data. The color space conversion unit 3201 converts the YCrCb data shown in FIG. 33 into NTSC-RGB data. FIG. 34 illustrates the data finally obtained by data expansion.
However, the above-described conventional ADCT compression method is an irreversible compression method in which data is lost during quantization in the sub-sampling unit 3102 and the quantization unit 3104. Thus, the expanded data differs from the data which is not yet subjected to compression. This is apparent from FIG. 34 which illustrates the NTSC-RGB data obtained by performing expansion. This means that degradation in the image quality has occurred.
Images created on a computer (computer created images), such as DTP data, have advantages in that they have sharp outlines and in that a single figure (or a single character) can be colored in a single color without noise being generated. However, the image data processed by the ADCT compression method suffers from disadvantages in that the outlines of a figure are not clear, in that a pseudo-edge (called mosquito noise) is generated, and in that colors change due to quantization, and thus loses the advantages inherent in the computer created image.
Particularly, since the above-described conventional compression is performed in the units of a block consisting of 8xc3x978 pixels, colors greatly change in a boundary area. Also, an increase in the compression rate causes the AC components to be finally lost, generating a block deformation and thus producing an image whose resolution has been reduced to one eighth of that of the original image.
In order to improve the image quality, image data may be compressed by a reversible compression method, such as the Huffman coding method. However, in this method, since the quantity of compressed data is variable, the quantity of compressed data to be stored in a memory cannot be fixed. Particularly, the processing of a gradation image or an image fitted into a computer created image and read by a scanner (hereinafter referred to as a natural image) by the reversible compression method may increase the quantity of data. Hence, a memory having a large capacity must be provided to prepare for the worst case.
The gradation or natural images have low-frequency components, unlike the normal computer created images having high-frequency components which are essential to the clear outlines and single color. Therefore, reversibility of the compressed/expanded data is not important to the gradation or natural images, and changes in the data due to the ADCT compression/expansion process may not degrade the image quality.
Further, although the amount of compressed data can be adjusted in the ADCT compression method, adjustment is performed by trial-and-error, i.e., by performing compression as a pre-scanning and then by performing compression again after a parameter for compression has been re-set on the basis of the quantity of compressed data obtained by pre-scanning. Thus, it takes much time to complete compression.
In brief, when the gradation or natural images are compressed by the reversible compression method, the compression rate cannot be increased. In addition, even when the ADCT compression method is employed, different compression methods may be employed for the portion of an image where degradation in the image quality is not apparent and for the other portion.
However, in the method in which the area is divided into blocks for the ADCT processing, as described above, a problem occurs when a block which contains a computer created image and a natural image in a mixed state (i.e., mixed together) is processed. That is, reversible compression performed on such a block increases the quantity of data, while ADCT compression performed on such a block deteriorates the image quality.
Furthermore, there are two types of compression method for multilevel color images: the reversible coding method and the irreversible coding method. Although a high compression rate cannot be obtained in the reversible coding method, deterioration in the decoded image does not occur, and the same image as the original image can thus be obtained. More specifically, when this reversible coding method is applied to a natural image which contains a large amount of halftone portions, it is impossible to obtain a high compression rate, although deterioration in the reproduced image does not occur. When the reversible coding method is employed for CG images, such as characters or figures created by, for example, the DTP software, since the number of colors which forms the image is less than that of a natural image, a relatively high compression rate can be obtained.
Although a high compression rate can be obtained in the irreversible coding method, deterioration in the decoded image occurs. The ADCT coding method which employs the DCT transform as the orthogonal transform has recently been drawing attention as an irreversible coding method. In this method, processing is performed in the units of a block having a certain size, and the DCT transform is performed at the initial stage. Since the DCT transformed coefficients are a frequency space, they are focused on a low frequency space, although focusing depends on an objective image. The frequency space is quantized, and the quantized coefficients are coded to obtain run length codes. Thus, when this ADCT coding method is applied to a natural image, since the natural image is highly focused on a low frequency space, a high compression rate can be obtained. In addition, deterioration in the decoded image is less. On the contrary, since the CG images, such as characters or figures created, for example, the DTP software, contain much high-frequency components, the degree of focusing in the frequency space is low. Therefore, when this ADCT coding method is used for coding the CG images, a high compression rate cannot be readily obtained. An increase in the compression rate may greatly deteriorate the quality of a decoded image.
Thus, a suitable coding method differs depending on the type of an objective image. Consequently, if a single coding method is employed for an image which contains both types of images, a natural image and a CG image, the image portion which is not suited to that coding method greatly deteriorates or a high compression rate cannot be obtained. Hence, the present inventors have filed Japanese Patent Application No. 4-217038 which discloses the coding method employing a plurality of methods for a single image. In this coding method, a CG image is coded by a reversible coding method suited to that image, while a natural image is coded by an irreversible coding method suited to that image. For a block in which there are two types of images, the two methods are employed for coding.
However, when the reversible coding method is employed for a mixed block, since the contour portion of a character or figure contains a high-frequency component, the image quality of that portion greatly deteriorates. Therefore, the character or figure portion of an objective block must be interpolated by natural image pixels which surround that character or figure portion so as to delete the high-frequency component of that block.
In addition, since this interpolation is performed synchronously with the operation of the coding unit which operates at a high speed, it must be able to be performed at a high speed.
Furthermore, even when an image (a natural image) fitted into a computer created image or a gradation image is processed by the reversible compression method, a high compression rate cannot be obtained. In some cases, the quantity of data may increase. The natural or gradation image has a low-frequency component, unlike a normal computer created image having a high-frequency component which is essential to the sharp outlines or singularity of color. Therefore, reversibility of the compressed/expanded data is not very important, and changes in the irreversibly compressed/expanded data do not generate apparent deterioration.
Thus, when the reversible compression method is employed for the natural or gradation images, a high compression rate cannot be obtained. Even when an irreversible compression method is employed, different compression methods must be employed for the image portion which does not deteriorate greatly and for the other image portion.
In view of the aforementioned problems of the prior art, an object of the present invention is to provide an image data compression apparatus which is capable of efficiently compressing or expanding image data, and a method thereof.
To achieve the above-described object, the present invention provides an image data compression apparatus which comprises irreversible compression means for compressing input multilevel image data for every block having a predetermined size by an irreversible compression method, and reversible compression means for compressing the input multilevel image data by a reversible compression method. Selection between output data of the irreversible compression means and output data of the reversible compression means is made for every pixel on the basis of data having a predetermined level and compressed by the reversible compression means.
Another object of the present invention is to provide an image data compression apparatus which enables the efficiency with which an image is compressed to be improved by substitution of image data.
To achieve the above-described object, the present invention provides an image data compression apparatus which comprises irreversible compression means for compressing input multilevel image data for every block having a predetermined size by an irreversible compression method, and reversible compression means for compressing the input multilevel image data by a reversible compression method. The reversible compression means performs compression after the input multilevel image data has been substituted on the basis of a predetermined reference.
Still another object of the present invention is to provide an image processing apparatus which enables interpolation of image data to be processed at a low cost and at a high speed.
To achieve the above-described object, the present invention provides an image processing apparatus in which a compression means is provided for each pixel type classified for every pixel in a block having a predetermined size, which comprises interpolation means for interpolating image data of pixels having an image type other than a predetermined image type by image data of pixels having the predetermined image type and located around the pixels having the image type other than the predetermined image type.
Still another object of the present invention is to provide an image processing apparatus which is capable of performing efficient compression by selecting a compression method according to changes in colors in a block.
To achieve the above-described object, the present invention provides an image processing apparatus which comprises irreversible compression means for compressing input multilevel image data for every block having a predetermined size by an irreversible compression method, reversible compression means for compressing the input multilevel image data by a reversible compression method, judgement means for judging changes in colors in the block on the basis of the input multilevel image data, and selection means for selecting either the irreversible compression means or the reversible compression means according to the result of the judgement of the judgement means.
Other objects and advantages of the present invention will become apparent from the following description taken in connection with the accompanying drawings.